Video processing system and video processing method

ABSTRACT

A system which allows significant camera breaks to be automatically detected among a large number of camera breaks and which is suitable in retrieving or playing a digital video is realized. Using a detected result recorded in an analysis result recording unit, a camera break analyzing unit compares the camera breaks among different digital videos. Thereby, the camera break analyzing unit detects camera breaks in which similar frames are detected, like frames used in common among a plurality of digital video of the same category. Such camera breaks are detected as the significant camera breaks among the large number of camera breaks and are used as indexes for retrieving or playing the digital video.

The present application is a continuation of U.S. application Ser. No.09/266,611, filed Mar. 11, 1999, the entire contents of which areincorporated herein by reference, which claims priority of JapaneseApplication serial number 10-061186, filed Mar. 12, 1998.

BACKGROUND

1. Field of the Invention

The present invention relates to a video processing system and a videoprocessing method and more particularly to a video processing system anda video processing method for automatically putting an index into avideo for retrieving or playing desired pictures.

2. Description of Related Art

Generally, a moving picture (a video) has an advantage that it can beaccessed randomly to a desired position and a picture (frame) byrecording it onto a CD-ROM, a CD-RW (ReWritable), a DVD (DigitalVersatile Disk)-ROM and a DVD-RAM and the like, instead of recordingonto media such as a magnetic tape from which a video can only be readin a sequential manner.

In order to access the desired positions and frames, it is necessary toinsert indexes among the video. However, it takes too much time to putthe indexes manually. With respect to marking the frames with theindexes automatically, there has been proposed a method of detectinglocations called a ‘camera break’ (where a different picture appears) bycomparing two frames temporally adjacent each other and putting in theindex.

However, such camera breaks appear approximately every five seconds invideos such as TV broadcasting programs and movies in general.Therefore, there arises a problem that if indexes are inserted to all ofthe detected camera breaks, the number of indexes increases excessivelyand it takes time to retrieve or play the desired pictures, which iscontrary to its purpose.

Accordingly, it is a purpose of the present invention to solve theabove-mentioned problems by providing a video processing system and avideo processing method which allow significant camera breaks to bespecified among a large number of camera breaks and which are suitablefor retrieving and playing the videos.

SUMMARY OF THE INVENTION

The present invention solves the above problems. In order to solve theabove-mentioned problems, the present invention provides a videoprocessing system for determining a desired camera break in a video,including a detecting unit for detecting at least one camera break inthe video, and an analyzing unit communicatively coupled to thedetecting unit, wherein the analyzing unit analyzes a level ofsimilarity between the detected camera break and a reference frame, andwherein the analyzing unit determines that the detected camera break isa desired camera break if the level of similarity is equal to or greaterthan a predetermined level.

According to a preferred embodiment of the present invention, thereference frame may be obtained from a different video. Both the videoand the different video may belong to a same category. The video may bea digital video.

In the system, the predetermined level may require that the referenceframe and the detected camera break be identical.

In the system, the detecting unit may detect a plurality of successivecamera breaks, wherein the analyzing unit may analyze a level ofsimilarity between the plurality of detected successive camera breaksand a plurality of reference frames, and wherein the analyzing unit maydetermine that the detected successive camera breaks are desired camerabreaks if the level of similarity is equal to or greater than apredetermined level.

The system may further includes a recording unit for recordinginformation concerning the desired camera break. The recordedinformation may include an index correspondence relationship between thedesired camera break and the reference frame.

The system may further includes a processing unit for processing thevideo using the recorded information. The processing unit may retrievethe desired camera break from the video. The processing unit may playsection in the video, wherein the section includes the desired camerabreak.

The system may further includes an other analyzing unit electricallyconnected to the recording unit, wherein the other analyzing unit mayanalyze a level of similarity between an other detected camera break andthe desired camera break stored in the recording unit, and wherein theother analyzing unit may determine that the other detected camera breakis an other desired camera break if the level of similarity is equal toor greater than an other predetermined level. Here, the predeterminedlevel may require that the reference frame and the detected camera breakbe identical.

The system may further includes an other recording unit for recordinginformation on the other desired camera break. The recorded informationmay include an index correspondence relationship between the desiredcamera break and the other desired camera break. The indexcorrespondence relationship may include information on a frame number ofthe desired camera break and the other desired camera break.

The video processing system may determine a desired camera break among aplurality of videos, wherein the detecting unit may detect at least onecamera break in each of the plurality of videos, wherein the analyzingunit may determine a number of videos among the plurality of videos inwhich the detected camera break of one of the videos has a level ofsimilarity to the other detected camera breaks of the other videos equalto or greater than a predetermined level, and wherein the analyzing unitmay determine that the detected camera break of the one video is adesired camera break if the number of videos is greater than apredetermined value.

The present invention also provides a video processing method fordetermining a desired camera break in a video, including steps ofdetecting at least one camera break in the video, analyzing a level ofsimilarity between the detected camera break and a reference frame, anddetermining that the detected camera break is a desired camera break ifthe level of similarity is equal to or greater than a predeterminedlevel.

According to a preferred embodiment of the present invention, thereference frame may be obtained from a different video. Both the videoand the different video may belong to a same category. The video may bea digital video.

In the method, the predetermined level may require that the referenceframe and the detected camera break be identical.

In the method, the detecting step may detect a plurality of successivecamera breaks, wherein the analyzing step may analyze a level ofsimilarity between the plurality of detected successive camera breaksand a plurality of reference frames, and wherein the analyzing step maydetermine that the detected successive camera breaks are desired camerabreaks if the level of similarity is equal to or greater than apredetermined level.

The method may further includes a recording step that recordsinformation concerning the desired camera break. The recordedinformation may include an index correspondence relationship between thedesired camera break and the reference frame.

The method may further includes a processing step that process the videousing the recorded information. The processing unit may retrieve thedesired camera break from the video. The processing unit may playsection in the video, wherein the section includes the desired camerabreak.

The method may further includes an other analyzing step that may analyzea level of similarity between an other detected camera break and thedesired camera break stored in the recording unit, and wherein the otheranalyzing step may determine that the other detected camera break is another desired camera break if the level of similarity is equal to orgreater than an other predetermined level. Here, the predetermined levelmay require that the reference frame and the detected camera break beidentical.

The method may further includes an other recording step that recordsinformation on the other desired camera break. The recorded informationmay include an index correspondence relationship between the desiredcamera break and the other desired camera break. The indexcorrespondence relationship may include information on a frame number ofthe desired camera break and the other desired camera break.

The video processing method may determine a desired camera break among aplurality of videos, wherein the detecting step may detect at least onecamera break in each of the plurality of videos, wherein the analyzingstep may determine a number of videos among the plurality of videos inwhich the detected camera break of one of the videos has a level ofsimilarity to the other detected camera breaks of the other videos equalto or greater than a predetermined level, and wherein the analyzing stepmay determine that the detected camera break of the one video is adesired camera break if the number of videos is greater than apredetermined value.

The present invention further provides a recording medium on which avideo processing program is stored for determining a desired camerabreak in a video, the video processing program being for causing acomputer to execute steps of detecting a camera break in the video,analyzing a level of similarity between the detected camera break and areference frame, and determining that the detected camera break is adesired camera break if the level of similarity is equal to or greaterthan a predetermined level.

The present invention yet provides a video processing system fordetermining a desired camera break in a video, including a detectingunit for detecting at least one camera break in the video, and ananalyzing unit communicatively coupled to the detecting unit, whereinthe analyzing unit analyzes a level of similarity between the detectedcamera break and a reference frame, and wherein the analyzing unitdetermines that the detected camera break is a desired camera break ifthe detected camera break and the reference frame are similar.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the system structure of a videoprocessing system according to one embodiment of the present invention.

FIG. 2 is a first diagram for explaining one way of analyzing camerabreaks among frames according to the embodiment.

FIG. 3 is a second diagram for explaining one way of analyzing thecamera breaks among the frames according to the embodiment.

FIG. 4 is a flowchart showing the procedure of one way of analyzing thecamera breaks among the frames according to the embodiment.

FIG. 5 is a diagram showing an example of a correspondence relationshipof the frames of a plurality of videos according to the embodiment.

FIG. 6 is an example showing index information obtained through theanalysis of the camera breaks in the embodiment.

FIG. 7 is a flowchart showing a procedure of a process for analyzing anew video of the same category by using camera breaks already detectedin the embodiment.

FIG. 8 is a diagram showing one way of analyzing the new videos of thesame category in the embodiment.

FIG. 9 is a flowchart showing a procedure of retrieving or playing avideo by using the result of the analysis of the camera breaks of theembodiment.

FIG. 10 is a diagram showing a way of detecting throughout a videosimilar frame sections by using the analysis of the camera breaks in theembodiment.

FIG. 11 is a flowchart showing a procedure of retrieving or playing avideo by utilizing the process for detecting throughout a video similarframes by using the analysis of the camera breaks of the embodiment.

FIG. 12 is a perspective view of a computer system for reading out avideo processing program from a recording medium according to anembodiment of the present invention.

FIG. 13 is a block diagram of the computer system of FIG. 12.

DESCRIPTION OF PREFERRED EMBODIMENTS

An embodiment of a video processing system and a video processing methodof the present invention will now be described in more detail withreference to the accompanying drawings.

FIG. 1 shows the system structure of a video processing system 100according to an embodiment of the present invention. This system isarranged so as to record digital moving pictures (digital video) onto arandomly accessible recording medium (a CD-ROM, a CD-RW, a DVD-ROM and aDVD-RAM, etc.), and to retrieve or play pictures of the digital video byusing the recording medium, and performs a function of analyzing videos.The system analyzes ‘camera breaks’ among a large number of frames of avideo by comparing a plurality of videos of the same category. Thereby,the system can automatically mark the frames with a significant indexand can retrieve specific frames, or play the video with skipping thespecific frames. Note that the present invention is not limited toretrieve or play only the digital videos, but also applicable toanalogue videos.

As shown in FIG. 1, a digital video processing unit #1 of the videoprocessing system includes a digital video recording unit 101, a camerabreak detecting unit 102, a detection result recording unit 103, acamera break analyzing unit 104 and an analysis result recording unit105.

The digital video recording unit 101 records a plurality of digitalvideos and is realized by using randomly accessible recording media asexplained above. The recording media may be a hard disk, a videotape ora semiconductor memory.

In the present embodiment, a plurality of digital videos of the samecategory are recorded in the digital video recording unit 101, andindexes for retrieving or playing such digital videos are created bysampling a feature common to those digital videos. Here, the digitalvideos of the same category mean those videos that contain at leastcontaining one identical common frame, e.g., a group of recorded videosof a specific type of TV news program that is broadcast every day at 7A.M., a group of recorded videos of a specific type of TV variety showprogram that is broadcast once a week, and the like. In these TVprograms, common video sources are often used at the beginning and atthe ending of each program. Specifically, the variety show program isoften composed of several sub-programs and the common video source isused in the opening of each sub-program.

The camera break detecting unit 102 detects camera breaks by taking outthe digital videos one by one from the digital video recording unit 101and compares frames temporally adjacent each other within the digitalvideos. In this case, although adjacent frames within one scene havehigh temporal correlation and are similar each other, the frames largelychange before and after the camera break and there is almost nocorrelation between the frames. The camera break is detected based onsuch non-similarity between frames that are temporally adjacent to oneanother.

In specific, a method disclosed in Japanese Patent Publication (Kokoku)No. 05-074273, for example, may be used in detecting the camera break(cut). According to this method, the camera break is detected bycalculating, for example, a change of luminance and a change offrequency of color per frame statistically in a recorded frame sequence,and by sampling the relative changes. The camera breaks include one typeof break in which the frame is changed instantly another type of breakin which the frame is changed gradually by a fade-in and fade-outtechnique called a ‘resolve’. In case of the resolve, a frame just afterwhen the scene has been changed completely is detected. The resolve maybe detected by the method introduced in Yukinobu Taniguchi, YoshinobuTomura and Hiroshi Hamada, “A method for detecting shot changes and itsapplication to access interfaces to video”, The Transactions of theInstitute of Electronics, Information and Communication for EngineersD-II, Vol. J79-D-II, No. 4, pp. 538-546, 1996, April, for example.

The camera breaks of each digital video detected by the camera breakdetecting unit 102 are recorded in the detection result recording unit103. Information recorded therein is the sequential numbers of thedetected frames. The corresponding frames of the recorded sequentialnumbers may be also recorded therein together. The recording media maybe a hard disk, a DVD-RAM, a videotape or a semiconductor memory. Thedigital video recording unit 101 may also be used as the recording unit103 in common.

Using the detection result recorded in the detection result recordingunit 103, the camera break analyzing unit 104 compares the frames of thecamera breaks among different digital videos to calculate a frequency ofthe appearance of sufficiently similar frames, i.e., frames that have alevel of similarity of at least a predetermined level, from the digitalvideos. Thereby, it detects the position of the camera break of thesimilar frames detected frequently from the other digital videos, suchas a frame used in common in a plurality of digital videos of the samecategory. The analysis result of the camera break analyzing unit 104 isrecorded in the analysis result recording unit 105 and is used as anindex for retrieving or playing the digital video on the digital videorecording unit 101. The information stored in the analysis resultrecording unit 105 is also utilized as template frames to be used ascomparative images in newly processing further digital videos of thesame category. The digital video recording unit 101 may also be used asthe analysis result recording unit 105 in common.

As described above, the significant camera break corresponding to thepoint of change of the program structure can be detected among a largenumber of camera breaks by analyzing the camera breaks using the digitalvideo processing unit #1.

A digital video processing unit #2 in FIG. 1 performs an image analyzingprocess on a new digital video by using the contents of the analysisresult recording unit 105 of the digital video processing unit #1 whenanother new digital video of the same category is to be newly processed.

That is, the digital video processing unit #2 takes out the new digitalvideo to be processed from a digital video recording unit 111 anddetects a camera break by a camera break detecting unit 112 by comparingthe temporally adjacent frames. A sequential number of the detectedcamera break is recorded in a detection result recording unit 113. Acamera break analyzing unit 114 compares frames of each camera break ofthe new digital videos to be processed with the template frames storedin the analysis result recording unit 105 to determine whether or notthey are sufficiently similar. The sequential numbers of the framessimilar to the template frames are recorded in an output resultrecording unit 115 as indexes of the new digital videos to be processed.

It is noted that the digital video recording unit 111, the camera breakdetecting unit 112 and the detection result recording unit 113 of thedigital video processing unit #2 have the same functions as the digitalvideo recording unit 101, the camera break detecting unit 102 and thedetection result recording unit 103 of the digital video processing unit#1, respectively. Therefore, the digital video processing unit #1 andthe digital video processing unit #2 may be realized as independentsystems or as a common system. When they are realized as a commonsystem, the units performing the same function may be shared among thedigital video processing units #1 and #2.

Next, the process performed by the camera break analyzing unit 104 willbe explained by using conceptual diagrams in FIGS. 2 and 3.

Here, videos of the same category are processed as videos 201 and 202 ofthe same category, such as a news program broadcast every day at 7 A.M.,or a variety show program put on the air once a week, for example. Asthese videos are in the same category, the length thereof is basicallythe same. However, because the camera break analyzing unit 104 can set aspecific range to be processed, the length of the videos to be processedneeds not always be the same. Frames 203 and 205 of the respectivecamera breaks detected by the camera break detecting process aredetected at many locations within the sequence of the videos in both ofthe videos 201 and 202. Normally, the camera breaks are detectedapproximately every five seconds.

As described before, the frames 204 are of the same image and usedregularly, i.e., title frame pictures of the opening and eachsub-program, will exist within the videos of the same category. In manycases, the same frames appear almost at the same time along the timedirection. FIG. 2 shows a case when the same frames appear at the sametime, and FIG. 3 shows a case when the same frames appear at temporallydifferent frame positions.

As for sub-programs which start at fixed times within a program, it maybe processed by limiting a time width to be analyzed. In other varietyshow programs, a sub-program is set irregularly, no sub-program is setor the sub-program is set with a considerable temporal deviation. Insuch a case, the system checks whether or not the same frames 204 existin the other digital videos by checking all combinations when there isno knowledge about it in advance.

Next, the procedure for analyzing digital videos will be explained withreference to FIG. 4. Here, suppose a case when five digital videos “NO.1” through “NO. 5” are recorded in the digital video recording unit 101as shown in FIG. 5. These five digital videos “NO. 1” through “NO. 5”are moving pictures of the same news program broadcast at differentdates.

At first, after detecting camera breaks in these five digital videos,the camera breaks are analyzed by the following procedure.

That is, it is first determined whether or not an unprocessed camerabreak exists in one digital video to be processed (suppose the video NO.1 here) in Step 301. When an unprocessed camera break exists, it iscompared respectively with camera breaks obtained from the other videosNO. 2 through NO. 4 of the same category in Step 302. When it isdetermined that the camera break within the video NO. 1 is sufficientlysimilar, such as identical, with one camera break contained in any oneof the other camera breaks NO. 2 through NO. 4 in Step 302, the indexcorrespondence relationship between these camera breaks within thevideos is recorded in Step 303.

Successively, the process returns to Step 301 to determine whether ornot another unprocessed camera break exists. When there are no moreunprocessed camera breaks, the process advances to Step 304 to count anumber of the same frames existing in the other videos with respect toeach of the camera breaks of the digital video to be processed. Next, acamera break that occurs more than a certain threshold value ofoccurrences among the digital videos is detected as a similar frameamong the camera breaks within the digital video to be processed in Step305. Then, this detected result is recorded in the recording unit as anindex in Step 306.

Because the common source images are used at the opening program and theweather forecast program in the example in FIG. 5, it is detected thatamong camera breaks A1 through A8 detected from the digital video NO. 1,the camera break A3 is the same frame with a camera break B3 within thedigital video NO. 2, as well as a camera break C4 in the digital videoNO. 3 and a camera break D4 in the digital video NO. 4. Accordingly, thepicture of the camera break A3 is determined to have a similarity withother videos of the same category with a high frequency. Similar tothat, it is detected that the camera break A8 within the digital videoNO. 1 is the same frame with camera breaks B8, C9 and D10 in the otherdigital videos, and the picture of the camera break A8 is determined tohave a similarity with high frequency.

In this case, among the camera breaks within the digital video NO. 1 tobe processed, the camera breaks A3 and A8 are selected as significantcamera breaks (i.e., desired camera breaks) and their positions are usedas indexes for retrieving or playing the videos.

It is also possible to generate indexes as shown in FIG. 6 in Step 306.

FIG. 6 shows two indexes (INDEX 1, INDEX 2) corresponding to detectedtwo similar frames. INDEX 1 indicates the digital videos and framenumbers where the common opening title of a program exists, and INDEX 2indicates the digital videos and frame numbers where the title of theweather forecast exists.

It becomes possible to retrieve or play only the weather forecastprogram from the plurality of digital videos one after another, forexample, by creating the indexes which can be used in common among theplurality of digital videos as shown in FIG. 6.

Next, with reference to the flowchart in FIG. 7, a process for using theanalysis result obtained through the process shown in FIG. 4 (the camerabreak obtained through the process in FIG. 4 will be referred to as a‘template frame’ hereinafter) to detect an identical camera break withthe template frame out of a digital video of the same category which isto be newly processed will be explained.

Here, FIG. 8 shows a case of analyzing a new video NO. 5 of the samecategory by using the analysis result of the videos NO. 1 through NO. 4which have been already analyzed.

At first, it is determined whether or not an unprocessed camera breakframe exists in Step 401. When an unprocessed camera break exists, thecamera breaks obtained from the video NO. 5 to be newly processed iscompared to the template frame in Step 402. When it is determined thatthe camera break and the template frame are the same (identical) frame,the correspondence relationship between these frames is recorded in Step403. Then, the process returns to Step 401 to either continue theprocess in Step 402 when there exists an unprocessed template frame, orto record the analysis result in Step 404 when no more unprocessedtemplate frames exist.

In the case of FIG. 8, two camera breaks E4 and E10 which are the sameframes as the template frames among the camera breaks in the new videoNO. 5 are selected as significant camera breaks. This is done by takingout the template frames one by one and by comparing them with the camerabreaks obtained from the new video NO. 5. Their positions are used asindexes for retrieving or playing the video NO. 5.

It is also possible to add information “Video NO. 5, Frame E4” to theINDEX 1 in FIG. 6 and also “Video NO. 5, Frame E10” to the INDEX 2.Thereby, the indexes common to the five digital videos are generated.Further, in the case when a user sets in advance a certain title, e.g.,‘Opening’, ‘Weather Forecast’, etc., indicative of that picture of thetemplate frame, it is possible to realize controls such that the briefprogram structure is displayed on a screen per each digital video byusing an icon or the like with a sub-program name (‘Opening’, ‘WeatherForecast’, etc.) or a corresponding video is automatically played byclicking the icon by linking the icon with the corresponding video inadvance.

It is noted that although the resultant template frames detected in theprocess in FIG. 4 are used in the process in FIG. 7, the template framesmay be provided manually in advance.

Further, it is not necessary to compare all camera breaks within a videoand the range of the search may be limited. For instance, when a camerabreak to be detected is generally set in a certain time slot of thevideo, the processing time may be shortened and the detecting accuracymay be improved by applying the comparison only to that time slot.

As for the method for comparing the camera breaks in general, a methodof considering the camera breaks is necessary to determine whether ornot they are considered or deemed to be identical. One approach that maybe applied is when an absolute sum of differences of RGB values of eachimage between the frames is less than a certain threshold value.However, because it takes time to obtain differences of all pixels, itis possible to use a method of comparing frames roughly by using a smallnumber of pixels randomly selected from the frames to remove those thatare apparently different and to select frames to be further compared,and then determining whether or not they are the identical frames bycomparing the whole pixels precisely at this point in time. Thereby, theprocessing speed may be higher.

FIG. 9 shows a procedure for retrieving or playing the video by usingthe analysis result of FIG. 7.

That is, at first in Step 501, it is determined whether or not anunprocessed template frame exists among the template frames, whichtemplate frames are given through the process in FIG. 4 or through themanual operation. When an unprocessed template frame exists, thetemplate frame is compared with respective camera breaks obtained fromthe video to be retrieved or played in Step 502. When it is determinedto be the same frame, their correspondence relationship is recorded inStep 503. Then, the process returns to Step 501 to continue the processin Step 502 when there exists an unprocessed template frame, orotherwise to record the analysis result in Step 504. Then, based on thisanalysis result, the digital video is retrieved or played.

In a type of news program through which current news is broadcast bit bybit together with old news, for example, there is a strong request foronly variant news, that is, news different from news already seen. Inthis case, only the newly added news may be viewed by using the alreadyviewed news program as a reference video, by sampling camera breaks fromthe viewed news program, by detecting similar frames from the new newsby utilizing the camera breaks as the aforementioned template frames andby generating and playing the new digital video so as to omit theportion where the similar frame exists.

FIG. 10 shows a case of detecting the similar frames and of retrievingor playing the video by using the same.

In FIG. 10, in case when the Video NO. 1 is a program already viewed andthe Video NO. 2 is a program to be newly viewed, particular sections IN1through OUT1, IN2 through OUT2 and IN3 through OUT3, where similarframes appear successively among the Video NO. 1 and the Video NO. 2,are detected. Video NO. 2 is retrieved or played by skipping thosesections. FIG. 11 shows a processing procedure of this method.

That is, all camera breaks of the Video NO. 1 are used as templateframes to compare with respective camera breaks of the Video NO. 2sequentially and the correspondence relationship of the frames detectedto be similar frames is recorded (Steps 601 through 603 of FIG. 11).Then, sections of the continuous or successive camera breaks within theVideo NO. 2 detected to be the similar frames through the analysis ofthe recorded result in Step 602 are detected in Step 605. The controlfor retrieving or playing the Video NO. 2 is carried out by using thedetected sections in Step 606.

Similarly, in case when the Video NO. 4 is an already viewed program andthe Video NO. 5 is a program to be newly viewed, sections where similarframes appear continuously in the Video NO. 4 and the Video NO. 5 aredetected, and the Video NO. 5 is retrieved or played by skipping thosesections.

On the other hand, it is also possible to retrieve the identical frames.In the example in FIG. 10, a region containing a large number of sameframes continuously may be specified by using the camera breaks of thesection IN1 through OUT1 of the Video NO. 1 sequentially as templateframes to compare with respective camera breaks of the other Videos NO.2 through NO. 5 to detect the position of the same frames.

Accordingly, a digital video similar to or the same as a certain digitalvideo may be found from a database in which a large number of digitalvideos are stored by comparing a camera break detected from a referencedigital video with camera breaks of the large number of digital videos,and by retrieving or playing a digital video containing a large numberof similar frames.

As described above, the present invention makes it possible to allows asignificant camera break corresponding to a point of change in thestructure of a whole program to be used as an index among a large numberof camera breaks by recording the frame position of the camera breakwhere a similar frame has been detected frequently. The presentinvention allows the program to be retrieved or played readily bycategorizing the program into sub-programs and the like. Further,because the camera breaks are compared with each other, a processingamount may be reduced considerably as compared to the case of comparingall frames of the videos. Accordingly, the significant camera breaks maybe detected through the process carried out by software by using ageneral-purpose computer, without using dedicated hardware. Therefore,all of the processes in FIGS. 4, 7, 9 and 11 may be realized by acomputer program and the same effect with the present embodiment may beobtained by executing the program on a computer, where the program maybe introduced to the computer through a recording medium such as afloppy disk or a CD-ROM.

A recording medium according to the present invention on which a videoprocessing program is recorded will be described with reference to FIGS.12 and 13. A recording medium according to the present invention onwhich the video processing program is recorded is read out by arecording medium driving device that is provided in a computer system asshown in FIG. 12, and used for video processing in a video processingsystem. As shown in FIG. 12, the computer system 50 has a computer mainbody 51 that is accommodated in a chassis such as a mini-tower, adisplay device 52 such as a CRT (cathode-ray tube), a plasma display, ora LCD (liquid crystal display), a printer 53 as a record output device,a keyboard and a mouse as input devices, a floppy disk drive 56, and anoptical disk drive 57.

FIG. 13 is a block diagram showing the above configuration. In additionto the computer main body 51, the chassis accommodates an internalmemory 55 such as a RAM (random access memory), and an external memorysuch as a hard disk drive unit 58. As shown in FIG. 12, the videoprocessing program that is, for example, recorded on a floppy disk 61can be read out by using a proper application program when the floppydisk 61 is inserted in a slot. The recording medium on which a programis recorded is not limited to the floppy disk 61, and may be a CD-ROM62. Other examples of the recording medium are MO (Magneto-Opticaldisk), an optical disk, and a DVD-ROM, all of which are not shown inFIG. 12.

The video processing program that is recorded on the recording mediumand can be read out by a computer or the like is one obtained bywriting, for instance, the steps of FIGS. 1,4,7,9 and 11 in a computerprogram format.

Further, although the present invention has been explained in the caseof analyzing a digital video which is what a user has obtainedautomatically through recording or broadcasting to retrieve or play thedigital video, the invention is also applicable to a case of creating avideo title composed of a plurality of digital videos of the samecategory. In such a case, various retrieval/play control using the indexcan be realized by recording the index as shown in FIG. 6 on the samerecording medium with the plurality of digital videos.

As described above, according to the present invention, a significantcamera break may be automatically detected among a large number ofcamera breaks, and indexes can be created and various retrieval or playcontrol processes may be performed by using the detected result.

It should be noted that the present invention should not exclusively belimited to a respective one of the illustrative embodiments and may bemodified into various forms without departing from the spirit and scopeof the invention.

1. A video processing system that determines portions of a video to record, comprising: a detector that detects at least one frame in a first video; an analyzer communicatively coupled to the detector and configured to analyze a level of similarity between the detected frame and a reference frame, wherein the reference frame is provided from a second video belonging to a same category as the first video; and a recorder communicatively coupled to the analyzer and configured to record a portion of the first video including the detected frame when the level of similarity is equal to or greater than a predetermined level.
 2. The video processing system according to claim 1, wherein the recorder begins recording at the detected frame when the level of similarity is equal to or greater than a predetermined level, and wherein the analyzer is further configured to provide an instruction to the recorder to stop recording the first video when the level of similarity between another frame of the first video occurring after the at least one frame and another reference frame is equal to or greater than the predetermined level.
 3. The video processing system according to claim 1, wherein the first video is a digital video.
 4. The video processing system according to claim 1, wherein the predetermined level requires that the reference frame and the detected frame be identical.
 5. The video processing system according to claim 4, the predetermined level requires that the reference frame and the detected frame be identical in video content.
 6. The video processing system according to claim 1, wherein the detector detects a plurality of frames in the first video, wherein the analyzer analyzes a level of similarity between the plurality of detected frames and a plurality of reference frames, the reference frames being provided from the second video, and wherein the analyzer determines that the detected frames correspond to points in the first video where the recorder is to either start or stop recording, when the level of similarity is equal to or greater than the predetermined level.
 7. The video processing system according to claim 1, wherein the detected frame is a camera break in the first video.
 8. A video processing method for determining which portions of a video are to be recorded, comprising: detecting at least one frame in a first video;. analyzing a level of similarity between the detected frame and a reference frame, wherein the reference frame is provided from a second video belonging to a same category as the first video; and recording a portion of the first video including the detected frame when the level of similarity is equal to or greater than a predetermined level.
 9. The video processing method according to claim 8, wherein the recording step comprises starting recording at the detected frame when the level of similarity is equal to or greater than a predetermined level, and wherein the video processing method further comprises stopping recording the first video when the level of similarity between another frame of the first video occurring after the at least one frame and another reference frame is equal to or greater than the predetermined level.
 10. The video processing method according to claim 9, wherein the first video is a digital video.
 11. The video processing method according to claim 10, wherein the predetermined level requires that the reference frame and the detected frame be identical.
 12. The video processing method according to claim 11, the predetermined level requires that the reference frame and the detected frame be identical in video content.
 13. The video processing method according to claim 8, wherein the detecting step comprises detecting a plurality of frames in the first video, wherein the analyzing step comprises analyzing a level of similarity between the plurality of detected frames and a plurality of reference frames, the reference frames being provided from the second video, and wherein the analyzing step further comprises determining that the detected frames correspond to points in the first video where the recording is to either start or stop recording, when the level of similarity is equal to or greater than the predetermined level.
 14. The video processing method according to claim 8, wherein the detected frame is a camera break in the first video.
 15. A recording medium on which a video processing program is stored for determining a desired location in a video, the video processing program being configured to cause a computer to execute the steps of: detecting at least one frame in a first video; analyzing a level of similarity between the detected frame and a reference frame, wherein the reference frame is provided from a second video belonging to a same category as the first video; and recording a portion of the first video including the detected frame when the level of similarity is equal to or greater than a predetermined level.
 16. The recording medium according to claim 15, wherein the video processing program is further configured to cause a computer to stop recording the first video when the level of similarity between another frame of the first video occurring after the at least one frame and another reference frame is equal to or greater than the predetermined level.
 17. The recording medium according to claim 16, wherein the first video is a digital video.
 18. The recording medium according to claim 17, wherein the predetermined level requires that the reference frame and the detected frame be identical.
 19. The recording medium according to claim 18, the predetermined level requires that the reference frame and the detected frame be identical in video content.
 20. The recording medium according to claim 15, wherein the detecting step comprises detecting a plurality of frames in the first video, wherein the analyzing step comprises analyzing a level of similarity between the plurality of detected frames and a plurality of reference frames, the reference frames being provided from the second video, and wherein the analyzing step further comprises determining that the detected frames correspond to points in the first video where the recording is to either start or stop recording, when the level of similarity is equal to or greater than the predetermined level. 